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Abstract — Autonomous  Underwater  Vehicles  (AUV)  ca- 
pable  of  adapting  their  mission  planning  in  response  to 
sensor  data  have  applicability  in  a  variety  of  applications. 
For  example,  to  find  the  source  of  a  chemical  in  a  fluid  en¬ 
vironment,  an  AUV  could  search  a  region  until  it  detects 
the  chemical  plume,  then  steer  the  vehicle  based  on  real¬ 
time  chemical  detections  and  flow  information  to  maneu¬ 
ver  the  vehicle  toward  the  odor  source.  Behavior  based 
planning  (BBP)  is  a  reactive  mission  planning  approach 
that  is  ideally  suited  for  such  applications.  This  article 
reviews  behavior  based  planning,  describes  an  example 
BBP  designed  to  locate  the  deepest  location  within  a  test 
area,  and  includes  results  from  in-water  experiments  on 
a  REMUS  AUV.  This  experiment  was  a  test  exercise  in 
preparation  for  the  use  of  BBP  on  a  REMUS  class  vehicle 
to  trace  chemical  plumes  to  their  source. 

Keywords —  Autonomous  vehicles,  behavior  based  plan¬ 
ning,  gradient  descent. 

I.  Introduction 

Various  biological  entities  (e.g.,  moths,  mosquitos,  crabs, 
lobsters)  are  capable  of  tracking  chemical  plumes  to  the 
chemical  source  in  turbulent  fluid  flow  environments. 
Autonomous  Underwater  Vehicles  (AUV)  with  such  ca¬ 
pabilities  have  utility  in  scientific  and  military  applica¬ 
tions.  The  ONR  Chemical  Sensing  in  the  Marine  En¬ 
vironment  (CSME)  and  ONR/DARPA  Chemical  Plume 
Tracing  (CPT)  programs  are  intended  to  develop  and 
demonstrate  such  capabilities.  Efficient  solution  of  the 
chemical  plume  tracing  mission  requires  the  AUV  to  have 
the  ability  to  reactively  change  the  mission  plan  (i.e.,  ve¬ 
hicle  trajectory)  in  response  to  sensor  information  (e.g. 
chemical  detection).  Preliminary  plume  tracing  test  re¬ 
sults  from  the  CSME  program  are  described  in  a  com¬ 
panion  paper  [1].  This  article  focuses  on  preliminary 
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testing  of  a  reactive  planning  methodology  known  as  be¬ 
havior  based  planning  (BBP). 

On- vehicle  in-water  testing  was  accomplished  on  a  RE¬ 
MUS  vehicle.  The  REMUS  was  modified  to  contain 
a  second  PC- 104  computer  to  implement  the  adaptive 
mission  planning  (AMP)  algorithms.  The  AMP  com¬ 
puter  is  connected  to  the  main  REMUS  computer  via 
serial  port.  When  enabled,  the  AMP  has  access  to  RE¬ 
MUS  state  and  sensor  data  and  can  issue  altitude/depth, 
speed,  and  heading  commands  to  the  REMUS  control 
system.  The  on-vehicle  test  and  demonstration  phase 
of  the  CSME  program  was  split  into  three  steps:  (1) 
AMP  enables  and  executes  simple  heading,  speed,  and 
depth  commands;  (2)  AMP  enables  and  executes  a  reac¬ 
tive  mission;  (3)  AMP  enables  and  executes  CPT  algo¬ 
rithms.  Step  1  was  completed  in  August  2002.  The  goal 
of  Step  1  was  to  test  the  communication  protocols  be¬ 
tween  the  main  REMUS  CPU  and  AMP.  The  objective 
of  Step  2  was  to  debug  and  test  the  ability  of  AMP  to  re¬ 
actively  control  the  REMUS  for  completion  of  a  mission. 
Due  to  the  time  and  expense  related  to  creating  surro¬ 
gate  chemical  plumes,  we  chose  an  artificial  mission  goal 
that  was  inexpensive,  amenable  to  solution  by  a  BBP 
approach,  and  required  extensive  reactive  control  of  the 
vehicle  for  its  solution.  The  test  mission  that  we  focused 
on  for  Step  2  was  finding  the  deepest  location  within  a 
specified  operating  area.  The  behavior  based  approach, 
algorithm  details,  and  in-water  results  are  described  in 
the  subsequent  sections. 
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II.  Behavior  Based  Planning 

In  the  late  1970’s  and  early  80’s,  Michael  Arbib  be¬ 
gan  to  investigate  models  of  animal  intelligence  from  the 
biological  and  cognitive  sciences  in  the  hopes  of  gaining 
insight  into  what  was  missing  in  robotics  [3].  At  nearly 
the  same  time,  Valentine  Braitenberg  studied  how  ma¬ 
chine  intelligence  could  be  evolved  by  using  sensor-motor 
pairs  to  design  vehicle  systems  [4] .  Later,  a  new  genera¬ 
tion  of  AI  researchers  began  exploring  the  biological  sci¬ 
ences  in  search  of  new  organizing  principles  and  methods 
of  obtaining  intelligence  in  robotic  systems.  They  pro¬ 
posed  the  reactive  behavior-based  approach  to  designing 
robotic  systems.  A  behavior  is  a  mapping  of  sensory  in¬ 
puts  to  a  pattern  of  motor  actions,  which  then  are  used 
to  achieve  a  task  and  will  serves  as  the  fundamental  com¬ 
ponent  for  designing  intelligent  systems. 

Rodney  Brooks’  subsumption  architecture  is  the  most 
influential  of  the  purely  reactive  paradigms.  Its  basic 
idea  is  to  describe  a  complex  task  by  several  behav¬ 
iors  with  simple  features  [5].  Behavior-based  control, 
however,  has  two  significant  disadvantages:  Firstly,  it 
may  be  difficult  to  formulate  reactive  behaviors  quanti¬ 
tatively.  Secondly,  there  might  be  no  obvious  approach 
to  coordinating  conflicts  and  competition  among  differ¬ 
ent  reactive  behaviors  to  achieve  good  performance.  A 
usual  approach  to  implementing  behavior-based  control 
is  the  use  of  artificial  potential  fields  [6].  A  drawback 
to  this  approach  is  that  proper  coordination  of  the  reac¬ 
tive  behaviors  requires  significant  effort  during  the  design 
phase  to  test  and  adjust  potential  fields  parameters.  This 
is  especially  true  when  these  thresholds  depend  heav¬ 
ily  on  environmental  parameters.  Another  alternative  is 
to  use  artificial  neural  networks  (ANNs).  In  the  ANN 
approach,  however,  there  are  many  ’’circumstance  pat¬ 
terns”  to  be  trained.  Dangerous  driving  situations  may 
occur  when  the  robot  meets  circumstance  patterns  that 
have  not  yet  been  learned  [10].  In  [9],  [7],  [8],  fuzzy- 
logic-based  reactive  behavior  strategies  are  proposed  to 
improve  the  performance  of  robot  systems  in  two  ways: 
First,  reactive  behavior  is  formulated  by  fuzzy  sets  and  a 
rule  base,  and  second,  conflicts  and  competition  among 
different  types  of  reactive  behavior  are  coordinated  by 
fuzzy  reasoning. 

The  application  of  BBP  to  the  deepest  point  search 
problem  is  described  in  the  reminder  of  this  article.  The 
behaviors  described  in  the  following  sections  used  tradi¬ 
tional  logic  as  the  basis  for  switching  between  the  be¬ 
haviors.  In  addition,  there  is  one  additional  behavior  re¬ 


ferred  to  as  the  Cage  behavior.  The  purpose  of  the  Cage 
is  to  ensure  that  the  vehicle  does  not  leave  the  specified 
operating  area  (OpArea).  The  Cage  function  commands 
a  heading  that  will  steer  the  vehicle  towards  the  interior 
of  the  OpArea.  Switching  of  the  Cage  with  the  behav¬ 
iors  described  below  is  based  on  simple  fuzzy  logic.  If  the 
vehicle  is  outside  the  OpArea,  then  the  Cage  heading  is 
commanded.  If  the  vehicle  is  inside  the  OpArea,  near 
the  boundary,  and  headed  out,  then  a  weighted  average 
of  the  Cage  and  Deepest  Point  headings  are  issued.  As 
the  vehicle  gets  closer  to  the  boundary,  the  Cage  head¬ 
ing  is  weighted  more  heavily.  This  effectively  removes 
the  outward  component  from  the  vehicle  velocity.  When 
the  vehicle  is  inside  the  OpArea  and  sufficiently  far  from 
the  boundary,  then  the  weighting  of  the  Cage  heading  is 
zero. 

III.  Deepest  Point  Planner  Overview 

Finding  the  deepest  location  within  a  region  of  the  sea 
with  unknown  bathymetry  was  selected  as  a  test  mis¬ 
sion  for  reactive  on-line  planning.  The  problem  will  be 
solved  using  a  REMUS  class  vehicle  and  BBP.  Success 
in  this  mission  is  a  step  along  the  path  towards  solving 
the  plume  tracing  problem. 

The  deepest  point  problem  could  be  solved  by  per¬ 
forming  a  ladder  search  that  covers  the  entire  test  re¬ 
gion.  For  a  rectangular  test  region  with  length  L  and 
width  W,  a  lawn-mower  search  parallel  to  the  region 
length  with  spacing  di  between  search  legs,  the  search 
path  length  is  p  =  L  x  J^-.  Therefore,  the  whole  search 
time  is  t  =  ^  ,  where  v  is  the  speed  of  the  vehicle 

and  the  time  between  legs  has  been  neglected. 

Since  the  goal  is  to  find  the  deepest  point  and  the  bot¬ 
tom  is  continuous,  the  vehicle  search  efficiency  could  be 
improved  by  concentrating  on  deep  regions  and  neglect¬ 
ing  shallow  regions.  A  BBP  has  been  designed  to  imple¬ 
ment  and  test  this  idea.  The  BBP  will  develop  a  function 
that  locally  approximates  the  bathymetry  in  the  vicinity 
of  the  vehicle.  This  local  map  will  be  used  to  determine 
the  direction  of  travel  that  maximizes  the  rate  of  depth 
increase.  For  this  approach  to  succeed,  the  local  map 
must  be  kept  accurate  and  enough  of  the  region  must 
be  explored  to  ensure  that  the  global  maximum  depth 
(within  the  OpArea)  is  found.  In  the  BBP  approach, 
we  break  the  problem  down  into  simpler  and  more  well 
defined  tasks  (Behaviors).  Algorithms  are  designed  to 
implement  each  of  these  behaviors  and  a  switching  mech¬ 
anism  is  designed  to  select  the  most  appropriate  behavior 
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to  control  the  vehicle  at  each  time.  For  the  deepest  point 
problem  we  required  three  behaviors:  GoTo,  CurveFit, 
GradientFollow . 

The  GoTo  behavior  issues  the  guidance  (i.e.,  heading, 
speed,  and  depth)  commands  required  to  maneuver  the 
vehicle  to  a  specified  location.  The  CurveFit  behavior 
maneuvers  the  vehicle  appropriately  to  ensure  that  the 
solution  for  the  local  depth  model  parameters  is  well- 
conditioned.  The  GradientFollow  behavior  commands 
the  heading  corresponding  to  the  gradient  of  the  local 
bathymetric  fit. 

A  behavior  switching  diagram  is  shown  in  Fig.  1.  The 
switching  logic  is  as  follows: 

Goto  — >  CurveFit.  When  the  vehicle  is  within  distance 
R  of  the  specified  point,  the  BBP  switches  to  CurveFit. 
CurveFit  — >  GradientFollow.  When  the  vehicle  completes 
a  maneuver  designed  to  yield  a  well-conditioned  matrix 
inversion  in  the  curve  fit  procedure,  the  curve  fit  parame¬ 
ters  are  computed.  If  the  curve  fit  has  no  local  maximum 
or  if  that  maximum  is  sufficiently  far  from  both  the  ve¬ 
hicle  location  and  all  previous  points  on  a  list  of  verified 
local  maxima,  then  the  BBP  switches  to  GradientFol- 
lowing. 

CurveFit  — »  GoTo.  When  the  curve  fit  has  a  local  max¬ 
imum  and  that  maximum  is  sufficiently  near  either  the 
vehicle  location  or  a  previous  verified  local  maximum, 
then  the  vehicle  has  achieved  and  verified  the  local  max¬ 
imum  of  the  current  curve  fit.  The  local  maximum  is 
added  to  the  list  of  verified  local  maxima.  Then,  the 
BBP  switches  to  GoTo. 

GradientFollow  — >  CurveFit.  If  the  vehicle  is  following 
the  gradient  and  the  water  depth  is  decreasing,  then  ei¬ 
ther  the  vehicle  has  past  a  local  maximum  or  the  curve  fit 
is  wrong.  If  the  vehicle  location  is  not  near  any  point  on 
the  list  of  verified  local  maxima,  then  the  BBP  switches 
to  curve  fitting. 

GradientFollow  — >  GoTo.  If  the  vehicle  is  following  the 
gradient  and  the  water  depth  is  decreasing,  then  either 


Fig.  1.  Behavior  Switching  Diagram 


the  vehicle  has  past  a  local  maximum  or  the  curve  fit  is 
wrong.  If  the  vehicle  location  is  near  a  point  on  the  list 
of  verified  local  maxima,  then  the  gradient  has  lead  the 
vehicle  to  the  basin  of  attraction  of  a  previously  known 
local  maximum;  therefore,  the  BBP  switches  to  curve 
fitting. 

This  cycle  repeats  by  going  to  p  start  locations,  which 
have  been  selected  to  ensure  sufficient  coverage  of  the 
operating  area.  For  the  results  shown  herein,  p  =  4  and 
the  four  points  are  the  corners  of  the  operating  area. 
At  the  completion  of  the  p  cycles,  the  point  on  the  list 
of  verified  local  maxima  that  has  the  largest  depth  is 
declared  and  the  deepest  point. 

A.  Behavior:  GoTo 

At  the  start  of  the  mission,  the  AMP  has  a  list  of  p 
locations  ( xSi,ySi ),  i  €  [1  ,p\  from  which  it  will  initiate 
gradient  descent  search.  The  GoTo  behavior  is  used  to 
maneuver  the  vehicle  to  each  of  these  p  locations  to  ini¬ 
tialize  gradient  descent  search.  Each  gradient  descent 
will  end  at  a  local  maxima  within  the  OpArea.  We  keep 
a  list  of  the  local  maxima  as  (x i,yi),  i  €  [l,n].  There¬ 
fore,  at  the  end  of  p  gradient  descent  trials  we  will  have 
a  list  of  n  <  p  local  maxima.  The  greatest  of  these  is  de¬ 
clared  as  the  deepest  point.  Then  the  mission  concludes. 

When  the  on-line  planner  is  in  GoTo,  the  output  of 
the  planner  is  the  heading  command 

V’c  =  arctan (y(t)  -  ySi,x(t)  -  xSi)  (1) 

where  (x(t),y(t))  is  the  current  vehicle  position  and 
(. xSi,ySi )  is  the  present  destination  location.  When  the 
vehicle  is  within  a  small  region  of  the  destination  loca¬ 
tion, 

V (a:(i)  -  xSi)2  +  ( y(t )  -  ySi )2  <  R  (2) 

the  planner  switches  from  GoTo  to  CurveFit. 

B.  Behavior:  CurveFit 

The  depth  of  the  sea  d  is  a  function  of  position  (x,y): 

d  =  /( x,  y)  (3) 

Since  this  function  is  not  known,  we  will  use  on-line  data 
to  estimate  a  second  order  Taylor  series  approximation 
to  the  depth  data: 

d(x,y,Xo,yo)  =  «i  +  aidx  +  a^dy  +  a^(dx)2 

+a5(dx)(dy)  +  ae{dy)2  (4) 

where  dx  =  (x  —  xo)  and  dy  =  (y  —  y o).  This  model  is 
locally  accurate  in  the  vicinity  of  (x’o,yo)-  The  second 
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order  a  approximation  d(x,y,  Xo,yo)  will  provide  a  gra¬ 
dient  direction  for  maximal  increase  of  d  and  an  estimate 
of  the  location  of  a  local  maxima. 

For  convenience  of  notation,  define 

d(x,y,x0,yo)  =  h(x,y,x0,y0)a  (5) 


Y  =  10  seconds.  Then  the  heading  command  is  changed 
by  90  deg.  and  held  constant  for  another  ^  =  10  sec¬ 
onds.  This  right  angle  trajectory  ensures  that  the  matrix 
H'H  is  sufficiently  well  conditioned.  At  the  end  of  N 
seconds,  the  CurveFit  uses  eqn.  (9)  to  estimate  a.  The 
extreme  point  of  eqn.  (8),  if  it  exists,  is  at 


where 


h  =  [1,  dx,  dy,  dx2,  (dx)(dy),  dy2] 

a  =  [or,  a2,  03,  Cl4)  05)  ■ 


The  local  model  is  specified  by  values  of  ( Xo,yo )  and  h. 
These  parameters  will  be  computed  from  a  set  of  data 
{(xi,  yi,  di)}YL1  where  {(xi,  yi)}f=l  are  within  a  small  re¬ 
gion  of  interest  near  the  current  vehicle  location  and  di 
represents  the  measured  water  depth  at  vehicle  location 
(xi,yt). 

The  linearization  point  is  selected  as  the  center  of  the 
available  dataset 

1  N  1  N 
xo=N^Xi  Vo=N^Vi'  (6) 

i=l  2=1 


Equation  (5)  is  a  linear  in  the  parameters  a.  Therefore, 
the  available  set  of  data  can  be  organized  into  a  ma¬ 
trix  equation.  Let  dx i  =  ( Xi  —  Xq),  dyi  =  (yi  —  yo),  d  = 
[dx,  d2,  •••  dN]'  and  hj  =  [1,  dxi,  dyu  dxf ,  (dxi)(dyi),  dyf] 
Then,  with  the  definition 


hi 


H  = 


hjv 


(7) 


we  have 

d  =  Ha  (8) 


where  a  is  the  only  unknown.  The  parameter  vector  a 
can  be  estimated  as 


dx *  = 


a3a5  —  2  a2a6 


,  *  a2a5-2a3a4  ,  , 

dy  =  -yr— : - -2“-  (10) 


4a4a6  —  a§  4a4a6  —  a§ 

This  point  is  a  local  maximum  if  04  <  0  and  06  <  0.  If  a 
local  maximum  exists,  the  location  of  the  local  maximum 
is  (x*,y*)  =  (x0  +  dx*,  yo  +  dy*). 

Now  we  compare  the  curve  fit  local  maximum  (x* ,  y*) 
with  the  list  of  previously  verified  local  maxima  (xi,yi), 
i  €  [l,n],  the  current  vehicle  location,  and  the  deepest 
point  encountered  thus  far  (xd,  yd)-  If  any  of  these  match 
to  accuracy  R,  then  we  assume  we  have  found  a  new  local 
maxima.  If  the  local  maximum  is  new,  we  store  add  it 
to  the  list  and  switch  to  G0T0.  If  there  is  no  match,  the 
BBP  switches  to  GradientFollow. 


C.  Behavior:  GradientFollow 

After  completion  of  the  CurveFit  behavior,  we  have  a 
locally  accurate  estimated  depth  function  d(x,  y)  and  an 
estimated  local  maximum  (x*,y*).  The  GradientFollow 
behavior  is  designed  to  verify  the  local  maximum  depth 
position.  Since  the  gradient  is  the  direction  in  which 
depth  increases  the  fastest,  the  heading  is  commanded 
in  the  direction  that  causes  gradient  following.  From 
equation  (4),  the  gradient  ( Gx,Gy )  at  position  ( x,y )  is 

dF 

Gx  =  —  =  a2  +  2a4(a:  -  a;0)  +  a5(y  -  y0)  (11) 

dF 

Gy  =  —  =  a3  +  2a6(y  -  y0)  +  a5(x  -  £0)  (12) 

dy 

The  corresponding  heading  command  is 

xpc  =  arctan(Gy,Gx).  (13) 


a  =  (H/H)_1H,d.  (9) 

Reliable  solution  of  eqn.  (8)  requires  that  the  matrix 
H'H  be  well-conditioned.  The  condition  number  of  ma¬ 
trix  H'H  depends  on  variation  of  the  rows  of  H,  which 
is  determined  by  the  sensed  positions  (xi,yi)  i  £  [l,iV]. 
If  the  set  of  points  in  {(x*,  yi)}f=1  are  along  a  line,  then 
the  condition  number  will  be  large  and  matrix  H'H  is 
ill-conditioned. 

For  this  set  of  experiments,  N  =  20  and  data  (ay,  y.-,,  dt) 
is  saved  at  a  1  Hz  rate.  Upon  switching  to  the  Curve- 
Fit  behavior,  the  heading  command  is  held  constant  for 


While  gradient  following,  the  depth  should  be  increased. 
If  the  depth  does  not  increase  over  a  T  second  interval, 
then  the  current  curve  fit  is  no  longer  accurate  and  the 
BBP  switches  back  to  the  CurveFit  behavior.  If  the  vehi¬ 
cle  successfully  arrives  near  (x*,y*),  then  the  estimated 
local  deepest  point  has  been  verified.  That  point  is  added 
to  the  list  of  local  maxima  and  the  BBP  switches  to  the 
G0T0  behavior. 

IV.  In- water  Results 

These  algorithms  were  tested  on  a  REMUS  class  vehi¬ 
cle  in  South  San  Diego  Bay  in  August  2002.  Two  runs 
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Fig.  2.  A  portion  of  the  vehicle  trajectory  (solid  line  going  through 
points  A,  B,  H)  overlayed  onto  a  post-mission  reconstruc¬ 
tion  of  the  bottom  contour  map  (colored  lines  with  numbers 
indicating  depth  in  meters). 

were  completed  over  the  same  operational  area.  The 
deepest  points  declared  on  the  two  runs  were  (32:37. 9273N, 
117:07. 1744W)  and  (32:37.9206N,  117:07. 1787W)  which 
match  to  within  13  m. 

Figure  2  displays  a  portion  of  the  vehicle  trajectory 
from  the  second  experiment  that  was  completed.  Prior 
to  this  segment  of  the  mission,  AMP  has  just  directed 
the  vehicle  to  GoTo  point  A.  The  vehicle  is  performing  a 
CurveFit  between  points  A  and  B.  AMP  performs  Gra- 
dientFollowing  of  the  curve  fit  between  points  B  and  C. 
Between  C  and  D,  AMP  is  performing  a  new  CurveFit. 
This  curve  fit  satisfies  the  stopping  conditions,  so  AMP 
transitions  to  the  next  staring  point  indicated  by  E.  Be¬ 
tween  E  and  F,  AMP  is  performing  a  new  CurveFit. 
AMP  is  GradientFollowing  between  F  and  G.  Between 
G  and  H,  AMP  is  CurveFitting. 

V.  Conclusion 

This  paper  has  reviewed  the  ideas  of  behavior  based 
planning  and  described  an  application  intended  to  im¬ 
plement  and  test  behavior  based  planning  on  a  REMUS 
class  vehicle.  The  selected  application  was  finding  the 
deepest  point  within  an  operational  area  using  gradient 
following  and  local  curve  fitting.  The  main  test  results 
indicate  (1)  that  the  AMP  algorithms  were  computable 
within  the  computational  constraints  of  the  vehicle,  (2) 
that  AMP  was  able  to  effectively  communicate  with  the 
REMUS  control  computer  to  achieve  the  trajectories  de¬ 
sired  for  solving  the  problem,  and  (3)  that  the  AMP 


algorithms  succeeded  in  accurately  finding  the  deepest 
point  of  the  region. 

This  deepest  point  application  was  not  of  interest  in 
its  own  right.  Our  main  interest  is  in  solving  problems 
related  to  Chemical  Plume  Tracing.  Due  to  the  cost  of 
creating  chemical  plumes  of  significant  length,  the  deep¬ 
est  point  application  was  selected  as  a  lower  cost  test 
problem  to  ensure  that  the  BBP  algorithms  were  imple¬ 
ment  able  on  the  vehicle  and  that  the  vehicle  was  control¬ 
lable  by  AMP  prior  to  actual  tests  related  to  chemical 
plume  tracing. 
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